Interactive Point-Based Modeling from Dense Color and Sparse Depth

We are developing a system for interactive modeling of real world scenes. The acquisition device consists of a video camera enhanced with an attached laser system. As the operator sweeps the scene, the device acquires dense color and sparse depth frames that are registered and merged into a point-based model. The evolving model is rendered continually to provide immediate operator feedback. This paper discusses interactive modeling of structured scenes, which consist of large smooth surfaces. We have built an acquisition device that captures 7x7 evenly spaced depth samples per frame. The samples are grouped into patches that are approximated with polynomial surfaces. Consecutive frames are registered by computing a motion that aligns their depth and color samples. The scene is modeled as a collection of depth images created on demand during scanning. Resampling errors are avoided by using offsets to record accurately the positions of the acquired samples. The interactive modeling pipeline runs at five frames per second.